Fastest Way to Read Text, Image, and Video in Python

Posted on | In ,

When comparing the efficiency of different libraries, there may exist a few orders of magnitude difference. In the implementation in demand of high efficiency, locate the time-consuming function and replace it with the most efficient library function.

  1. Text: Pandas
    Installation: pip install pandas or conda install pandas

    1
    2
    import pandas as pd
    data = pd.read_csv(text_name, sep=',', header=None)

  2. Image: Pillow-SIMD, skimage, OpenCV, imageio

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    import cv2
    import skimage
    import imageio
    from PIL import Image

    #read a 1280x720 image
    pil_im = Image.open(image_name) #0.0057s
    pil_im = pil_im.resize((448,448)) #0.025s

    sk_im = skimage.io.imread(image_name) #0.026s
    sk_im = skimage.transform.resize(sk_im, (448, 448)) #0.060S

    cv_im = cv2.imread(image_name) #0.021s
    cv_im = cv2.resize(cv_im, (448, 448)) #0.0016s

    im = imageio.imread(image_name) #0.033s

    Pillow-SIMD is faster than Pillow, which is not reported here. OpenCV is the most efficient one here.

  3. Video: OpenCV, skvideo, imageio

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    import cv2
    import imageio
    import skvideo

    #read 30fps video with each frame 1280x720 
    cap = cv2.VideoCapture(video_name)
    ret, frame = cap.read() #0.002s

    vid = imageio.get_reader(video_name, 'ffmpeg')
    for image in vid.iter_data(): #0.004s

    skvideo.setFFmpegPath(os.path.dirname(sys.executable))
    videogen = skvideo.io.vreader(video_name)
    for img in videogen: #0.073s

    For OpenCV in Anaconda, it sometimes fails in reading from video but succeeds in reading from camera. In this case, /usr/bin/python is recommended. imageio and OpenCV are comparable here.